General Testing Fisher , Neyman , Pearson , and Bayes

One of the famous controversies in statistics is the dispute between Fisher and Neyman-Pearson about the proper way to conduct a test. Hubbard and Bayarri (2003) gave an excellent account of the issues involved in the controversy. Another famous controversy is between Fisher and almost all Bayesians. Fisher (1956) discussed one side of these controversies. Berger’s Fisher lecture attempted to create a consensus about testing; see Berger (2003). This article presents a simple example designed to clarify many of the issues in these controversies. Along the way many of the fundamental ideas of testing from all three perspectives are illustrated. The conclusion is that Fisherian testing is not a competitor to Neyman-Pearson (NP) or Bayesian testing because it examines a different problem. As with Berger and Wolpert (1984), I conclude that Bayesian testing is preferable to NP testing as a procedure for deciding between alternative hypotheses. The example involves data that have four possible outcomes, r = 1, 2, 3, 4. The distribution of the data depends on a parameter θ that takes on values θ = 0, 1, 2. The distributions are defined by their discrete densities f(r|θ) which are given in Table 1. In Section 2, f(r|0) is used to illustrate Fisherian testing. In Section 3, f(r|0) and f(r|2) are used to illustrate testing a simple null hypothesis versus a simple alternative hypothesis although Subsection 3.1 makes a brief reference to an NP test of f(r|1) versus f(r|2). Section 4 uses all three densities to illustrate testing a simple null versus a composite alternative. Section 5 discusses some issues that do not arise in this simple example. For those who want an explicit statement of the differences between Fisherian and NP testing, one appears at the beginning of Section 6 which also contains other conclusions and com-